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I INTRODUCTION 


A. OVERVIEW 

SRI International (SRI) operates an upper-atmosphere research facility in Spndre Strpmfjord 
(Sondrestrom), Greenland. This facility, which is supported by the National Science Foundation, 
contains a number of instruments, which have been used for independent experiments funded by 
a number of research organizations. The facility is built around an incoherent-scatter radar, a 
complex and expensive to operate instrument requiring an on-site crew of engineers and techni- 
cians. Other instruments at the site include: 

• A lidar system, operated by SRI. 

• An all-sky camera, operated by Lockheed Palo Alto Research Laboratory. 

• A Fabiy-Perot interferometer, operated by University of Michigan. 

• An imaging riometer, operated by University of Maryland and Danish Meteorological 
Institute (DMI). 

• A digisonde operated, by Air Force Phillips Laboratory, University of Lowell, and 
DMI. 

• An ozone spectrometer, operated by Cente National de la Recherche Scientifique 
France and DMI. 

• A Michelson interferometer, operated by Embry-Riddle University. 

Observational campaigns studying upper atmospheric and space phenomena have usually 
necessitated investigator visits to (or actual participation at) the facility. Because of the site’s 
remote location, travel to and from the facility is time consuming and expensive; with the closing 
of the U.S. Air Force Base in Sondrestrom, travel costs are expected to rise further. 

Research at Sondrestrom is also frequently collaborative, requiring use of multiple instru- 
ments at the site, and often requiring coordination with other sites. For example, SRI is a princi- 
pal investigator in NASA’s International Solar Terrestrial Physics (ISTP) program and often 
needs to coordinate operation of the radar with ISTP-related satellites. However, limited space 
and logistical support restrict the number of investigators that can visit the facility at a given 
time. 

B. MOTIVATION FOR A TELESCIENCE SYSTEM 

Providing remote access to Sondrestrom via a telescience capability is intended to increase 
acccess to data and to reduce the amount of time an investigator must spend at the site. Immedi- 
ately, it will allow SRI to quickly provide radar data to the ISTP database. Ultimately, it will 
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enable multiple investigators to discuss real-time data and experiments interactively from remote 
locations. This ability to “video conference” live experiments will reduce costs and increase the 
opportunities for collaboration, and will allow increased access of ongoing experiments to stu- 
dents and other passive observers. 

C. PROGRESS 

In the first year of this project, SRI established an electronic satellite link between NASA 
Goddard Space Flight Center and the Sondrestrom airport and developed display-software based 
on the X window system. The intent was to use the X window system protocol to transport dis- 
plays and user interface events between remote locations. The satellite link, while operational, 
had not been fully tested and debugged at the end of the first year. Thus, it had not been possible 
to undertake a full test of the first year’s accomplishments in a “live” situation. Instead, the 
software developed during the first year was tested using simulated data and using two comput- 
ers communicating at a 56 kilobaud rate. The results implied that the overhead associated with 
the X protocol is great enough that operating over a long-distance 56-kbaud network would result 
in unacceptably slow performance. 

The goals of the project covered by the second year’s funding have been to establish full 
connectivity between the radar facility and sites in the United States, to evaluate the performance 
of this link, and to develop telescience software that makes more effective use of the available 
network bandwidth and throughput. It was anticipated that the most efficient technique would be 
to transmit the data needed to generate a display, rather than the displays themselves (i.e., to 
avoid the overhead of the X protocol over the low-capacity, long-distance network). 
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II HARDWARE ARCHITECTURE 


Figure 1 shows a diagram of network connectivity to the Sondrestrom radar facility. A 
number of gateways that exist between SRI in Menlo Park and the Sondrestrom radar facility 
LAN are not shown in the Figure 1. Of particular importance to the discussion here are the links 
between the Sondrestrom ground station at the airport and the radar facility, and between 
Greenland and NASA Goddard (via satellite). At 56 kbaud, they are the lowest capacity links in 
the network and thus are the limiting factor. 

Figure 2 shows a diagram of the Sondrestrom radar facility LAN, a thin Ethernet connected 
to the Internet and occupied primarily by PCs* running disk operating system (DOS) and Novell 
Netware. The upper half of the figure depicts the primary computers associated with the radar: 

• DTO and DTI are responsible for data acquisition and for archiving data to the File 
Server using Netware 386. 

• DERT detects new data as they are written to the file server and displays various gray- 
scale and vector field plots of the data. 

Several systems that are not relevant to the current telescience effort have been omitted from 
Figure 2; these include the PC that controls pointing of the antenna and the computers associated 
with other scientific instruments (all of these are or will be PCs running DOS). A VAXStation 
3100 running VMS on the network provides the multitasking capability needed for telescience. 


*“PC” is used generically here to refer to IBM Personal Computer clones running MS-DOS. 
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FIGURE 1 NETWORK ARCHITECTURE RELEVANT TO SONDRESTROM TELESCIENCE CAPABILITY 


INCOHERENT-SCATTER RADAR COMPUTERS 



OTHER COMPUTERS 


FIGURE 2 SONDRESTROM HARDWARE ARCHITECTURE 
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Ill SOFTWARE ARCHITECTURE 


A. MOTIVATING ISSUES 

Several issues have guided software design. While it is desirable to build a system with 
future enhancements in mind, the limitations of the current hardware and software must also be 
taken into consideration. Existing limitations stem from two major factors : 

• The 56-kbaud network link between the Sondrestrom radar facility and the United 
States. 

• The preference of investigators for PCs running DOS for real-time data acquisition. 

While 56 kbaud is sufficient for transporting radar displays, it is inadequate to handle multiple 
simultaneous users. It is also insufficient for some anticipated future users; for example, the 
Lockheed all-sky camera can generate data rates of well over 1 Mbyte/s. Thus, the ability to 
eventually manage the traffic on the “weakest” link was given major consideration. 

Most investigators in the upper atmospheric science community use PC systems running 
DOS for three reasons: 

• They are familiar and comfortable with DOS. 

• Interrupts are handled in a reliable and timely manner in single- tasking environments, 
such as DOS, whereas the response time for interrupts in most UNIX systems is unpre- 
dictable. Reliable and timely interrupt handling is critical to real-time data acquisition. 

• PC systems running DOS are relatively inexpensive in comparison with the alternative 
choices. 

Typical instrument computers are either interrupt-driven or execute an event loop, which polls 
for interrupts. In either case, the computer must respond to an interrupt and execute a set of 
operations (e.g., archive data, write data to the network) within a specified period of time. To 
maintain continued real-time operation, the telescience software architecture must allow the 
instrument computer to control when data are written to the network. 

Single-tasking operating systems introduce similar constraints. Because these systems can- 
not run a server in background, means must be provided to “push” data through the network. 

B. DESCRIPTION 

The Sondrestrom telescience software is based on the client/server paradigm. It assumes no 
detailed knowledge about the content or structure of the data from each instrument. Thus, each 
instrument computer periodically writes a fully complete, time-tagged data object to the network. 
Figure 3 is a block diagram of the Sondrestrom telescience system software. 
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FIGURE 3 SONDRESTROM SOFTWARE ARCHITECTURE 


After initially establishing a connection, each instrument can write data objects to the 
instrument server running on the VAX. The instrument server writes the data object to disk, 
deleting any older objects from that particular instrument client. No archiving is currently per- 
formed; that is, only the most current data object is retained on disk. 

The VAX in Sondrestrom executes a “wire-service” server, which provides data to the VAX 
at SRI Menlo Park. After the client establishes a connection to the wire-service server, the server 
writes data objects to the client whenever they appear on disk. It is a client’s task to generate 
displays. 

This architecture has the following features : 

• An instrument client is never required to wait for the VAX before it can write data. 

• If needed, network traffic control can be built into the wire-service server, because it is 
at one end of the “weakest link.” 

• Data pertaining to graphics and displays are not explicitly sent over the network. The 
wire-service client must know the structure of the data objects sent over the network 
and must generate displays. 

• The instrument server can serve many clients simultaneously, provided that the data 
rates do not exceed the capacity of the Ethernet controller and disk systems on the 
VAX. Thus, adding scientific instruments is relatively easy. 

Note that any real-time data objects that are lost because of throughput limitations will disappear 
at the VAX in Sondrestrom. A possible future enhancement would be to perform archiving and 
retrieval of data objects on the VAX. 
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C. IMPLEMENTATION 


The instrument and wire-service servers were written using the Internet protocol suite. In 
particular, all data are transmitted over sockets using TCP (Transmission Control Protocol). TCP 
provides reliable delivery of data streams between two computers (unlike the faster UDP, where 
speed is achieved at the cost of reliability of datagram delivery). The VAX implementation was 
layered on top of the socket library interface provided with the TGV’s Multinet* Programmer’s 
Toolkit. Multinet sockets are source-level compatible with BSD sockets. Porting to UNIX plat- 
forms should be relatively easy. 

The PC instrument clients were written using the LAN Workplace for DOS Application 
Programming Interface (API). This API provides a socket library that is very similar to the BSD 
socket library. However, it is sufficiently different that a small effort will be required to port to a 
UNIX-based system or to a DOS system using an alternate API (e.g.. Sun’s PC-NFS program- 
ming library). 

The Sondrestrom telescience software consists of the following C language functions, 
which must be incorporated into any instrument client: 

establish_connection() Requests a connection to the instrument server and 

identifies the client to the server 

send_data() Sends data to the instrument server 

close_connection() Closes the connection. 

At present, these functions assume that the instrument PC has an Ethernet controller card made 
by Microdyne Corp. (the EXOS 205T ) and that it also has Novell’s LAN Workplace for DOS 
socket library API. Investigators that are not currently on the network are aware of these 
requirements and have planned appropriately. 

Similarly, there are three functions that enable a wire-service client to receive data: 

establish_connection() Requests a connection to the wire service server and 

identifies the client to the server 

get_data_set() Determines whether new data have been sent to the 

client; if so, accepts the new data object 

close_connection() Closes the connection. 

These functions were written using TGV’s Multinet socket programming library. The TGV 
socket library is very similar to BSD sockets available on most UNIX systems. If needed, porting 
to UNIX should be simple. 

At present, there is only one instrument client — the incoherent-scatter radar — and one wire 
service client — SRI Menlo Park. The software on the DERT PC (see Figure 2) was modified to 


* Multinet is a licensed/trademarked product of TGV, Inc. 
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include instrument client function calls. In addition to detecting new radar data objects and gen- 
erating displays for the Sondrestrom site, DERT now sends new data objects to the instrument 
server running on the Sondrestrom VAX. 

The wire-service client in Menlo Park runs on a VAX running VMS. It accepts radar data 
objects from the network and generates displays in Menlo Park. While these displays are similar 
to those on the DERT PC, they are generated using the X window system (as opposed to DOS- 
based graphics). All overhead associated with the X window system is limited to the LAN in 
Menlo Park. 

This implementation has the following features and limitations: 

• Adding new instruments is relatively simple. The instrument computer must be 
updated to include calls to the instrument client functions. Additionally, a wire-service 
client must be written to accept new data and generate displays. 

• At present, only one wire-service client is allowed. This limitation exists because of 
the desire to manage channel capacity. A future enhancement will be to write a server 
running in Menlo Park that will provide data objects to any number of clients on the 
Internet. 

D. RESULTS 

The two standard applications, File Transfer Protocol (ftp) and Telnet, were originally used 
to test the robustness and the throughput of the network connection between Menlo Park and 
Sondrestrom. While the weakest link in the connection is the 56 kbaud data rate, the actual per- 
formance is somewhat less than the theoretical maximum because of the overhead associated 
with data packets passing through various gateways and because of other traffic in the network as 
a whole. Telnet, or remote login, provides a simple and useful indications of latency in the con- 
nection, because each character that is typed is sent to the remote computer and then sent back to 
be displayed at the local machine. The ftp file transfer utility provides information on the actual 
possible throughput for sustained data transfers. 

Telnet shows a latency that ranges from imperceptible (with the human eye) to several sec- 
onds. For some issues relating to remote control of instruments, this is not acceptable. However, 
it is more than satisfactory for accessing and viewing data from instruments at Sondrestrom. 

File transfers between Menlo Park and Sondrestrom show an average throughput of 
1 kbyte/s for files ranging in size from 10 kbyte to 10 Mbyte with a throughput ranging from 800 
to 1300 byte/s. This transfer rate is significantly lower than the 56-kbaud limit represented by 
the network link because of network overhead and because of the ftp protocol overhead. The 
actual available throughput is sufficient to transmit some data from the radar and most other 
instruments at the site. 

We have tested and demonstrated the Sondrestrom telescience software on several occa- 
sions. Each test was performed during a scheduled incoherent-scatter radar experiment. 


8 



Investigators in Menlo Park were able to view the results of each scan in pseudo real time. The 
network throughput was adequate, as indicated by the fact that very few data objects were lost 
across the network. 

It was not possible to measure the latency reliably because time has not been precisely syn- 
chronized between the Sondrestrom and Menlo Park computer systems. However, we estimate 
that it is similar to that of Telnet, that is, mostly varying between one and ten seconds. This 
latency was apparent in Menlo Park as the time between display updates varied considerably. 
The time between updates of DERT displays for these experiments is usually nearly constant. 

To provide a degree of interactivity, we used the utility program TALK to communicate 
with the Sondrestrom site crew. TALK enables users on a network to carry on interactive con- 
versations. This allowed investigators in Menlo Park to view the most recent measurements and 
to discuss radar operation with the site crew. 
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IV CONCLUSIONS AND RECOMMENDATIONS 


The atmospheric physics community’s reaction to this project has been very positive. In 
addition to real-time monitoring of the incoherent-scatter radar displays, the electronic link has 
been used to transport data from Sondrestrom for inclusion in NASA’s ISTP database. A num- 
ber of investigators now wish to receive real-time data from Sondrestrom. The proposed uses of 
the connection include passive observation of the measurements, discussions of the measure- 
ments by geographically separated investigators, and remote control of instruments. These 
demands are highlighted by several workshops on technology for scientific collaboration funded 
by the National Science Foundation during the past year. 

While reaction to the project has been very positive, there remain some limitations 
attributable to existing hardware and software. It is clear that investigators’ demands will even- 
tually, if not quickly, exceed the existing system. Needed improvements and enhancements 
include the following: 

• A higher capacity electronic connection between Sondrestrom and the United States. 
We anticipate that existing and proposed uses will quickly exceed the capacity of the 
existing connection. 

• Software for managing data volume on the link between Sondrestrom and the United 
States. This should include some data compression/decompression, as well as capacity 
scheduling. 

• Data archival and retrieval software for Sondrestrom. Because of the harsh environ- 
ment in Greenland, the electronic connection has been unavailable at times. In such 
cases, archival would allow browsing of the data once network service resumes. 

• Software to support multi-investigator collaboration. This requires client and server 
software for distributing data from Menlo Park (or NASA Goddard) to multiple inves- 
tigators requesting data. 
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